Let’s examine how well the Tectonic system meets its non-functional requirements, as discussed in the first lesson.

Performance isolation#

We use many measures to ensure that each tenant (and their applications) get their requested resources, but they don’t impact others. Each tenant’s storage and IOPS needs profile is maintained using quotas initially. Tenants can go beyond their quota if free IOPS is available. Request throttling and fair queuing keep each tenant to its allocated resources.

Because of the exclusive storage allocation, tenants are physically isolated in terms of storage. We prioritize storage traffic using TrafficGroups and TrafficClasses. Doing so helps us provide low latency service to the applications that need them.

Data integrity#

Keeping the data integrity intact has been very crucial in large-scale systems. Since these systems face lots of data corruption, users want to ensure that it is not corrupted data they are accessing. In-memory data corruption is very common in such large-scale systems. To avoid this, we have used checksums to ensure data integrity.

The system ensures the integrity of the data by using checksums. For the verification of data in the memory, we use the following steps:

1. We take the data ($D$) and generate a checksum ($CS_{D}$).
2. We’ll perform an in-memory transformation ($T$) such that $D' = T(D)$.
3. We’ll convert transformed data ($D'$) back to original data ($D$) by using the inverse function ($I$) of transformation ($T$) to verify the data such that $D = I(D')$.
4. We’ll compare the checksum of inverted transformed data ($CS_{I(D')}$) with the checksum of original data ($CS_{D}$). If both are equal, data is not corrupted. Otherwise, data is corrupted.

Conclusion#

The Tectonic file system disaggregates metadata and data components of the system and delegates management of the metadata to a building block—ZippyDB. Additionally, user-level file naming, system-level file objects, and associated blocks and chunks are managed by different sub-systems so that each sub-system can scale independently. We can provide exabytes-level storage where disk space and IOPS are efficiently and fairly shared among applications of many tenants. Applications can choose from the Reed-Solomon encoding or full replication to meet their storage use versus latency tradeoffs.

System design wisdom in Tectonic#

• Tectonic is another example of carefully demarcating different system parts, where some parts can be delegated to a different service (we can think of them as building blocks). We used a well-managed, highly available key-value store to keep our metadata instead of inventing our own.

• If we compare Tectonic with the Colossus file system, we see many similarities. Tectonic uses ZippyDB, while Colossus used Bigtable (both key-value stores at their core) to store the file system’s metadata. It is surprising how two independent groups (Tectonic and Colossus) can converge on similar solutions.